Sparse Empirical Bayes Analysis (SEBA)

We consider a joint processing of n independent sparse regression problems. Each is based on a sample (yi1, xi1) . . . , (yim, xim) of m i.i.d. observations from yi1 = x T i1βi+εi1, yi1 ∈ R, xi1 ∈ R, i = 1, . . . , n, and εi1 ∼ N(0, σ), say. p is large enough so that the empirical risk minimizer is not consistent. We consider three possible extensions of the lasso estimator to deal with this problem, the lassoes, the group lasso and the RING lasso, each utilizing a different assumption how these problems are related. For each estimator we give a Bayesian interpretation, and we present both persistency analysis and non-asymptotic error bounds based on restricted eigenvalue type assumptions. “. . . and only a star or two set sparsedly in the vault of heaven; and you will find a sight as stimulating as the hoariest summit of the Alps.” R. L. Stevenson